Reactive Information Displays to Augment Comprehension and Problem Solving

In earlier research, Narayanan and Hegarty developed a model of how people comprehend multimodal information displays of systems in VSC domains. This model, although described in detail with empirical support elsewhere (Narayanan & Hegarty, 2002), is reiterated in the following paragraphs since it forms the theoretical foundation for RID designs that I will investigate in proposed research. The model views comprehension as a constructive process with six stages, through which the user integrates his or her prior knowledge of the domain with the presented information to build a mental model of the system that is being described. The resulting internal representation is a mental model that is " runnable " in that it contains information that allows the user to mentally simulate a system and generate predictions about its operation. This cognitive model postulates that people construct a mental model of a dynamic system by decomposing it into simpler components, retrieving relevant background knowledge about these components, and mentally encoding the relations (spatial and semantic) between components to construct a static mental model. They then mentally simulate this static mental model, beginning with some initial conditions and inferring the behaviours of components one by one in order of the chain of causality or logic. This mental process depends on prior knowledge (e.g. rules that govern the behaviour of the system in question), text comprehension skills, spatial visualization skills for encoding and inferring information from graphic displays, and ability to integrate information in multiple modalities. Mental model construction under these circumstances involves six stages of processing that are described next. Although these processes are listed sequentially, they are not necessarily accomplished in this order and during comprehension a user may iterate through these processes multiple times to elaborate his or her mental model of the system. Decomposition. A complex s ystem typically consists of individual components or elements. The user needs to parse the representations on the external display into units that correspond to meaningful elements of the domain. This decomposition process is probably guided both by prior knowledge about the system and its components (a top-down influence) and by visual properties of the external representations. Constructing a static mental model by making representational connections. Another stage in comprehension involves making representational connections among the visual and symbolic units identified during decomposition. This stage involves making two types of connections: (a) connections to prior knowledge and (b) connections